gh-116146: Add C-API to create module from spec and initfunc #139196

itamaro · 2025-09-21T07:18:54Z

C-API WG decision: capi-workgroup/decisions#77

Issue: Expose a C-API function to allow custom importers to create a module using an init function #116146

encukou

This looks reasonable.

I'm proposing a replacement for Init functions in PEP-793, but that PEP adds PyModule_FromSlotsAndSpec and PyModule_Exec which should work for similar use cases.
Would it make sense to switch the spec & initfunc arguments to match that proposal?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Do you want to submit this to the C API WG?

Python/import.c

itamaro · 2025-09-23T15:38:11Z

Would it make sense to switch the spec & initfunc arguments to match that proposal?

I think it should be fine! Is the PEP already implemented on main?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Yes, I used it to reflect what it does internally. PyImport_LoadModuleFromInitfuncAndSpec sounds great to me!

Do you want to submit this to the C API WG?

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

encukou · 2025-09-24T13:55:55Z

Is the PEP already implemented on main?

No, it's not even accepted :)
I have a slight preference to align the two proposals, in case both make it. I'm checking if you have strong reasons for the order here. No need to update the PR now.

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

Just go to the WG :)

itamaro · 2025-09-24T14:51:34Z

Just go to the WG :)

done :) capi-workgroup/decisions#77

Python/import.c

- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

Python/import.c

Co-authored-by: Kumar Aditya <[email protected]>

encukou · 2025-11-11T15:03:49Z

I left my suggestions as a PR: itamaro#27

* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <[email protected]>

Doc/c-api/import.rst

Include/cpython/import.h

Programs/_testembed.c

Python/import.c

Co-authored-by: Victor Stinner <[email protected]>

python-cla-bot · 2025-11-12T16:15:22Z

All commit authors signed the Contributor License Agreement.

…on under the free-threaded build

itamaro · 2025-11-12T16:20:26Z

thank you @encukou @vstinner @kumaraditya303 for the review and suggestions!

@encukou are you planning to merge gh-141197 soon? until that is merged, this PR has a docs failure due to referring to the function that you document in gh-141197.

vstinner

The new function PyImport_CreateModuleFromInitfunc() should be documented in Doc/whatsnew/3.15.rst.

Misc/NEWS.d/next/C_API/2025-11-08-10-51-50.gh-issue-116146.pCmx6L.rst

Doc/c-api/import.rst

Include/cpython/import.h

Programs/_testembed.c

Python/import.c

Co-authored-by: Petr Viktorin <[email protected]> Co-authored-by: Kumar Aditya <[email protected]> Co-authored-by: Victor Stinner <[email protected]>

vstinner

LGTM. Thanks for the updates!

vstinner · 2025-11-14T11:07:33Z

Congrats @itamaro!

bedevere-app bot mentioned this pull request Sep 21, 2025

Expose a C-API function to allow custom importers to create a module using an init function #116146

Closed

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from f9cd705 to 4363e5a Compare September 21, 2025 19:25

encukou mentioned this pull request Sep 23, 2025

_PyImport_SwapPackageContext would be helpful if exposed #139227

Open

encukou reviewed Sep 23, 2025

View reviewed changes

Python/import.c Outdated Show resolved Hide resolved

itamaro mentioned this pull request Sep 24, 2025

Expose a C-API to allow custom importers to create a module directly from spec & initfunc capi-workgroup/decisions#77

Closed

6 tasks

vstinner reviewed Sep 26, 2025

View reviewed changes

Python/import.c Outdated Show resolved Hide resolved

itamaro added 5 commits November 8, 2025 08:09

pythongh-116146: Add new C-API to create builtin from spec and initfunc

403c7e2

Apply review feedback and CAPI WG decision

074a510

- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

Add tests for the PyImport_CreateModuleFromInitfunc API

d38c0e4

Document the new PyImport_CreateModuleFromInitfunc C-API

d5be6e5

Add news entry

781a730

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 4363e5a to 781a730 Compare November 8, 2025 18:52

itamaro added 2 commits November 8, 2025 10:54

Fix lint

7af34eb

fix CI

35e9e78

- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

itamaro changed the title ~~gh-116146: Add new C-API to create builtin from spec and initfunc~~ gh-116146: Add C-API to create module from spec and initfunc Nov 9, 2025

itamaro marked this pull request as ready for review November 9, 2025 01:01

itamaro requested review from brettcannon, ericsnowcurrently, kumaraditya303, ncoghlan and warsaw as code owners November 9, 2025 01:01

bedevere-app bot added the awaiting review label Nov 9, 2025

kumaraditya303 reviewed Nov 9, 2025

View reviewed changes

Python/import.c Outdated Show resolved Hide resolved

Apply Kumar's review suggestion

6aa2fb2

Co-authored-by: Kumar Aditya <[email protected]>

PyImport_CreateModuleFromInitfunc: Doc & test update (#27)

f652399

* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <[email protected]>

vstinner reviewed Nov 12, 2025

View reviewed changes

Apply Victor's review suggestions

c4f1cc8

Co-authored-by: Victor Stinner <[email protected]>

itamaro added 2 commits November 12, 2025 08:17

Set error if initfunc is NULL

29b1b68

Account for the RuntimeWarning when import a singlephase init extensi…

1165950

…on under the free-threaded build

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 8f16533 to 1165950 Compare November 12, 2025 16:18

vstinner reviewed Nov 12, 2025

View reviewed changes

Misc/NEWS.d/next/C_API/2025-11-08-10-51-50.gh-issue-116146.pCmx6L.rst Outdated Show resolved Hide resolved

encukou reviewed Nov 13, 2025

View reviewed changes

Doc/c-api/import.rst Outdated Show resolved Hide resolved

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Include/cpython/import.h Outdated Show resolved Hide resolved

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Programs/_testembed.c Outdated Show resolved Hide resolved

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Python/import.c Outdated Show resolved Hide resolved

Apply suggestions from code review

79e5d0e

Co-authored-by: Petr Viktorin <[email protected]> Co-authored-by: Kumar Aditya <[email protected]> Co-authored-by: Victor Stinner <[email protected]>

itamaro requested a review from AA-Turner as a code owner November 13, 2025 15:31

Mention new C-API in whatsnew

fbfde0a

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from fabafb5 to fbfde0a Compare November 13, 2025 15:32

vstinner approved these changes Nov 13, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Nov 13, 2025

kumaraditya303 approved these changes Nov 13, 2025

View reviewed changes

Merge branch 'main' into pythongh-116146-create-builtin-initfunc

f485fcb

encukou merged commit 1e4e59b into python:main Nov 14, 2025
46 checks passed

bedevere-app bot removed the awaiting merge label Nov 14, 2025

encukou added a commit to encukou/cpython that referenced this pull request Nov 14, 2025

Link reference to PyModule_Exec added in python#139196

816023d

Uh oh!

gh-116146: Add C-API to create module from spec and initfunc #139196

gh-116146: Add C-API to create module from spec and initfunc #139196

Conversation

itamaro commented Sep 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

itamaro commented Sep 23, 2025

Uh oh!

encukou commented Sep 24, 2025

Uh oh!

itamaro commented Sep 24, 2025

Uh oh!

Uh oh!

Uh oh!

encukou commented Nov 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-bot bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

itamaro commented Nov 12, 2025

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

itamaro commented Sep 21, 2025 •

edited

Loading

python-cla-bot bot commented Nov 12, 2025 •

edited

Loading